AbiPy v0.9.1 @IMEC¶
M. Giantomassi and the AbiPy group¶
10th international ABINIT developer workshop
May 31 - June 4, 2021 - Smart Working, Lockdown@BE
- These slides have been generated using jupyter, nbconvert and revealjs
- The notebook can be downloaded from this github repo
- To install and configure the software, follow these installation instructions
Use the Space key to navigate through all slides and SHIFT + Space key to go back one slide.
What is AbiPy?¶
Python package for:¶
- Generating ABINIT input files automatically
- Post-processing results extracted from netcdf or text files
- Interfacing ABINIT with external tools such as vesta, wannier90, lobster, etc.
- Running ABINIT-specific workflows on laptops as well as on HPC clusters
Dependencies:¶
- Hard deps: pymatgen, netcdf4, matplotlib, plotly, numpy, scipy, pandas, ipython …
- Soft deps: ASE, phonopy, jupyter …
NB: AbiPy can be interfaced with other packages (e.g ASE, phonopy) via converters.
How to install AbiPy¶
Using conda and the conda forge channel (recommended):
conda install abipy --channel conda-forge
Since conda is not limited to python packages, one can install ABINIT in the same env with:
conda install abinit -c conda-forge
Using pip and python wheels:
pip install abipy --user
For further info see this installation howto
Jupyter notebooks with examples and lessons inspired by the official tutorials¶
In [4]:
%embed https://nbviewer.jupyter.org/github/abinit/abitutorials/blob/master/abitutorials/index.ipynb
Out[4]:
Since AbiPy is not restricted to high-throughput, we'll show how to use the terminal to analyze calculations
Well, a python script would be much more flexible but the goal here is to show that one can replace grep, vim, gnuplot with AbiPy
- Then we show how to use the python API for improved flexibility
No perl scripts were harmed in the making of this notebook
Command line interface¶
- abiopen.py ➝ Open output files inside ipython or print/visualize file
- abistruct.py ➝ Operate on crystalline structures read from file
- abicomp.py ➝ Compare multiple files (i.e. convergence studies)
- abiview.py ➝ Quick visualization of output files
- abinp.py ➝ Generate input files for typical calculations
- abigui.py ➝ Start GUI in web browser
Documentation¶
abistruct.py --helpfor manpageabistruct.py COMMAND --helpfor help aboutCOMMAND
HTML documentation available at http://abinit.github.io/abipy/scripts/index.html
Examples¶
abistruct.py spglib si_scf_GSR.nc
abistruct.py convert si_scf_GSR.nc -f cif
abiopen.py si_scf_GSR.nc --print
and many more...¶
abiopen.py
Print results stored in FILE (more than 45 file extensions supported)¶
In [5]:
!abiopen.py si_scf_GSR.nc --print
To produce a predefined set of matplotlib figures, use:¶
abiopen.py mgb2_kpath_FATBANDS.nc --expose --seaborn
Replace --expose with --notebook to generate a jupyter notebook with predefined python code¶
If you are still using perl scripts to extract data from the main output file, check this out:¶
abiopen.py gs_dfpt.abo --expose --seaborn
To visualize all the results inside the web browser, use¶
abiopen.py out_GRUNS.nc --expose-web # -ew for the short version
Creating dashboards from the command line¶
- Creating panels inside a notebook is great if you needs GUIs plus the possibility of executing python code.
- There are however cases in which we only need a dashboard with widgets to interact with the data.
Convert structure from netcdf format to CIF (abivars, xsf, poscar, qe, siesta, wannier90, json)¶
In [6]:
!abistruct.py convert si_scf_GSR.nc -f cif
Read structure from FILE and print ABINIT variables for NSCF along k-path:¶
In [7]:
!abistruct.py kpath si_scf_GSR.nc
Connect to the materials project database to find similar structures.¶
In [8]:
!abistruct.py mp_match si_scf_GSR.nc
Need to call anaddb to compute and visualize ph-bands and DOS from DDB?¶
abiview.py ddb ZnSe_hex_qpt_DDB --seaborn
Add --phononwebsite to visualize data on the phononwebsite by Henrique¶
In [9]:
!abiview.py --help
To compare multiple structures:¶
In [10]:
!abicomp.py structure *.cif si_nscf_GSR.nc *_DDB
AbiPy API¶
AbiPy plots with matplotlib¶
In [11]:
gsr = abiopen("si_nscf_GSR.nc")
gsr.ebands.plot(with_gaps=True);
AbiPy plots with plotly¶
In [12]:
gsr.ebands.plotly(with_gaps=True); # obj.plot becomes obj.plotly
To upload the plotly figure to the chart studio server, use:¶
gsr.ebands.plotly(with_gaps=True, chart_studio=True);
Users can finally customize the AbiPy plot without changing the python code 🎉¶
Interactive 3d plots with plotly:¶
In [13]:
gsr.ebands.kpoints.plotly(title="k-path in 3d with plotly");
and, ça va sans dire, phonons from DDB with plotly:¶
In [14]:
znse_ddb = abilab.abiopen("ZnSe_hex_qpt_DDB")
phbst_file, phdos_file = znse_ddb.anaget_phbst_and_phdos_files()
phbands, phdos = phbst_file.phbands, phdos_file.phdos
phbands.plotly_with_phdos(phdos, units="cm-1", title="ZnSe Phonon bands + DOS in cm-1");
In [15]:
paths = [
"mgb2_888k_0.01tsmear_DDB",
"mgb2_888k_0.04tsmear_DDB",
"mgb2_121212k_0.01tsmear_DDB",
"mgb2_121212k_0.04tsmear_DDB",
]
paths = [os.path.join(abidata.dirpath, "refs", "mgb2_phonons_nkpt_tsmear", f)
for f in paths]
robot = abilab.DdbRobot()
for i, path in enumerate(paths):
robot.add_file(path, path)
In [16]:
# Define function to change labels:
func = lambda ddb: "nkpt: %s, tsmear: %.2f" % (
ddb.header["nkpt"], ddb.header["tsmear"])
robot.remap_labels(func)
robot
Out[16]:
Now we can build a dataframe with the most important parameters:¶
In [17]:
robot.get_params_dataframe()
Out[17]:
and check that all DDBs have been computed with the same crystalline structure:¶
In [18]:
robot.get_lattice_dataframe()
Out[18]:
To analyze the effect of k-sampling/smearing on the vibrational properties:¶
In [19]:
# Invoke anaddb and store results
r = robot.anaget_phonon_plotters(nqsmall=2)
r.phbands_plotter.gridplot_with_hue("tsmear", with_dos=True);
How to run AbiPy Flows¶
⚠️ There are two different workflow infrastructures:¶
Internal AbiPy implementation (abipy.flowtk modules):¶
- ✅ Lightweigth, no database required
- ✅ Designed for rapid prototyping and/or for supporting advanced ABINIT capabilities
- ❌ No explicit support for high-throughput (HT) applications.
AbiFlows package (requires Fireworks and MongoDB database)¶
- ✅ HT-oriented: use MongoDB to store workflow status and results for further analysis
- ✅ High-level API designed for HT applications (e.g. phonon calculations for the materials project)
- ❌ Not all the flowtk workflows are immediately available in AbiFlows.
NB: There's an ongoing effort to reimplement AbiFlows in terms of the atomate framework. In this talk, we will mainly discuss the new features available in abipy.flowtk.
In order to run a Flow, we need two configurations files:¶
scheduler.yml providing:
- scheduler parameters such as days, hours, minutes, max_njobs_inqueue, max_ncores_used, …
manager.yml providing:
- list of shell commands to be executed before running ABINIT
- list of modules to load
- options for the queue manager (bluegene, moab, pbspro, sge, shell, slurm, torque)
See this page for examples or use the abidoc.py script and the syntax:
- abidoc.py scheduler
- abidoc.py manager
- abidoc.py manager slurm
In [ ]:
# %load ~/.abinit/abipy/manager.yml
qadapters:
-
priority: 1
queue:
qname: gmac
qtype: shell
job:
mpi_runner: mpirun
pre_run:
#- export OMP_NUM_THREADS=1
#- export PATH=$HOME/git_repos/abinit/_build/src/98_main:$PATH
- ulimit -n 2048
- source ~/env.sh
#- source ~/miniconda3/bin/activate test_abinit
#- export PATH=$HOME/git_repos/abinit/_build/src/98_main:$PATH
limits:
min_cores: 1
max_cores: 2
timelimit: 0:10:0
hardware:
num_nodes: 1
sockets_per_node: 1
cores_per_socket: 2
mem_per_node: 4 Gb
# Optional
#condition: {"$eq": {omp_threads: 2}}
In [ ]:
# %load ~/.abinit/abipy/scheduler.yml
# The launcher will stop submitting jobs when the
# number of jobs in the queue is >= Max number of jobs
max_njobs_inqueue: 2
# Maximum number of cores that can be used by the scheduler.
max_ncores_used: 2
# number of weeks to wait.
#weeks: 0
# number of days to wait.
#days: 0
# number of hours to wait.
#hours: 0
# number of minutes to wait.
#minutes: 0
# number of seconds to wait.
seconds: 8
# when to first execute the job and start the counter (default is after the given interval).
#start_date:
# Send mail to the specified address (accepts string or list of strings).
# PRO TIP: the scheduler WILL try to send and email after a default time of 4 days. If you
# comment out the mailto address, this will cause the scheduler to terminate, with
# potentially nefarious effects on your running jobs. If you do not wish to receive
# emails, a work around is to set the variable `remindme_s` below to something very
# large (say, 100 days).
#mailto: nobody@nowhere.com
# verbosity level (int, default 0)
#verbose: 0
# The scheduler will shutdown when the number of python exceptions is > max_num_pyexcs
#max_num_pyexcs: 2
# The scheduler will shutdown when the number of Abinit errors is > max_num_abierrs
#max_num_abierrs: 0
# The scheduler will shutdow when the total number of tasks launched is > safety_ratio * tot_num_tasks.
#safety_ratio: 5
# Send an e-mail to mailto every remindme_s seconds.
#remindme_s: 345600
abinp.py is handy for generating input files quickly but it cannot compete with the flexibility of the python interface:¶
In [20]:
def make_scf_input(ecut=2, ngkpt=(4, 4, 4)):
"""
Generate an `AbinitInput` to perform GS calculation for AlAs.
Args:
ecut: Cutoff energy in Ha.
ngkpt: k-mesh divisions
Return:
`AbinitInput` object
"""
gs_inp = abilab.AbinitInput(structure="AlAs.cif",
pseudos=["13al.pspnc", "33as.pspnc"])
# Set the value of the Abinit variables needed for GS runs.
gs_inp.set_vars(
nband=4,
ecut=ecut,
ngkpt=ngkpt,
nshiftk=4,
shiftk=[0.0, 0.0, 0.5, # This gives the usual fcc Monkhorst-Pack grid
0.0, 0.5, 0.0,
0.5, 0.0, 0.0,
0.5, 0.5, 0.5],
tolvrs=1.0e-10,
)
return gs_inp
Inside the notebook, one gets the HTML representation with links to the documentation:¶
In [21]:
make_scf_input()
Out[21]:
Abipy will call Abinit to get the list of DFPT perturbations and…¶
Moreover, from an AbiniInput one can easily build more complicate workflows:¶
In [22]:
def build_flow_alas_phonons():
"""
Build and return a Flow to compute the dynamical matrix on a (2, 2, 2) qmesh
as well as DDK and Born effective charges.
The final DDB with all perturbations will be merged automatically and placed
in the Flow `outdir` directory.
"""
from abipy import flowtk
scf_input = make_scf_input(ecut=6, ngkpt=(4, 4, 4))
return flowtk.PhononFlow.from_scf_input("flow_alas_phonons", scf_input,
ph_ngqpt=(2, 2, 2), with_becs=True)
In [23]:
flow_phbands = build_flow_alas_phonons()
flow_phbands.get_graphviz()
Out[23]:
Quick intro to AbiPy Flows and selected examples¶
Effective masses with DFPT¶
This flow performs:¶
- GS-SCF run followed by NSCF run with k-path to locate band edges automatically
Compute $\epsilon^{\alpha\beta}_{n\bf{k}}$ and the effective mass tensor at the band edges using $|u_{n\mathbf{k}}\rangle$, and the k-derivatives $|u_{n\mathbf{k}}^\alpha\rangle$ $H^\alpha_{\mathbf{k}}$, $H^{\alpha\beta}_{\mathbf{k}}$
For the formalism, see J. Laflamme Janssen, et. al. Phys. Rev. B 93, 205147
- More that 1k systems already computed by J. C. Abreu.
Python API:¶
flow = flowtk.Flow("flow_effmass_dfpt")
# Build input for GS SCF calculation.
scf_input = make_scf_input()
# This object implements all the worflow logic
from abipy.flowtk.effmass_works import EffMassAutoDFPTWork
work = EffMassAutoDFPTWork.from_scf_input(scf_input)
flow.register_work(work)
Dependency Graph¶
- To run the flow with the high-level interface, use: run_effmass_dfpt.py --sheduler
- autoparal algorithm:
- select "optimal" Ncpus for each task
- set MPI-related variables in the input
- submit tasks and start monitoring
- indipendent tasks are executed in parallel
- Support for bluegene, moab, pbspro, sge, shell, slurm, torque
- Restart capabilities (e.g. timeout limit) and error handlers
To generate a GS-SCF input, one can use factory functions (HT-oriented)¶
In [25]:
from abipy.abio.factories import gs_input
gs_input(structure="si.cif", pseudos="14si.pspnc", ecut=8)
Out[25]:
or build it from scratch if full controll is wanted:¶
In [26]:
other_inp = abilab.AbinitInput(structure="si.cif", pseudos="14si.pspnc")
other_inp.set_vars(ecut=8, nsppol=2, toldfe=1e-8)
other_inp.set_autokmesh(8)
other_inp
Out[26]:
How to run calculations?¶
The simplest way to start the scheduler from the shell is via the syntax:
run_elastic.py --scheduler # -s for the short option
For non-trivial Flows, we suggest to put the scheduler in background and use nohup
so that we can disconnect from the shell session without killing the scheduler.
nohup run_elastic.py --s > log 2> err &
Obviously, it is possible to submit a Slurm script that executes the script on the compute note with 1 core.
To interact with the Flow, one can also use the abirun.py script, e.g:
abirun.py FLOWDIR status